Papers with vector representation
VSP at PharmaCoNER 2019: Recognition of Pharmacological Substances, Compounds and Proteins with Recurrent Neural Networks in Spanish Clinical Cases (D19-57)
Copied to clipboard
| Challenge: | The Named Entity Recognition of drugs, medications and chemical entities in Spanish is a new task in the field of NLP . |
| Approach: | They propose to use SNOMED CT term search engine to classify the entities in Spanish and a neural model for the Named Entity Recognition. |
| Outcome: | The proposed system achieves 76.29% and 60.34% performance in the Named Entity Recognition and Concept indexing tasks. |
Transformer and seq2seq model for Paraphrase Generation (D19-56)
Copied to clipboard
| Challenge: | Existing methods for generating paraphrases fall into one of these broad categories -rule-based, seq2seq, deep generative models and a varied combination. |
| Approach: | They propose a framework that combines transformer and sequence-to-sequence models for better quality of generated paraphrases. |
| Outcome: | The proposed framework improves on two datasets-QUORA and MSCOCO using transformer and sequence-to-sequence models. |
LTV: Labeled Topic Vector (C18-2)
Copied to clipboard
| Challenge: | Using nnDDC, we generate labeled topic classifications based on the Dewey Decimal Classification (DDC) Unlike related approaches, we use classifiers to define the dimensions of CISS, which are directly labeles by the underlying target class. |
| Approach: | They propose a website and API that generates labeled topic classifications based on the Dewey Decimal Classification (DDC) they propose nnDDC, a largely language-independent natural network-based classifier for DDC, which is language-dependent . |
| Outcome: | The proposed model is language-independent and performs well in 40 languages. |
Representation of Lexical Stylistic Features in Language Models’ Embedding Space (2023.starsem-1)
Copied to clipboard
| Challenge: | lexical stylistic notions such as complexity, formality, and figurativeness can be identified in pretrained Language Models . static embeddings encode these features more accurately at the level of words and phrases whereas contextualized LMs perform better on sentences. |
| Approach: | They propose to derive a vector representation for stylistic notions from seed pairs . they find that static embeddings encode stylistic features more accurately . |
| Outcome: | The proposed representations can be used to characterize new texts in terms of these dimensions using a small number of seed pairs. |
Norm-Based Curriculum Learning for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
| Approach: | They propose a norm-based curriculum learning method that measures difficulty, competence and weight of a sentence in a word embedding. |
| Outcome: | The proposed method outperforms baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
Entity Commonsense Representation for Neural Abstractive Summarization (N18-1)
Copied to clipboard
| Challenge: | Current ELS’s are not sufficiently effective, possibly introducing unresolved ambiguities and irrelevant entities. |
| Approach: | They propose an off-the-shelf entity linking system to extract linked entities and propose Entity2Topic (E2T) module attachable to a sequence-to-sequence model that transforms a list of entities into a vector representation of the topic of the summary. |
| Outcome: | The proposed model improves the performance of the Gigaword and CNN summarization datasets by at least 2 ROUGE points. |
Optimizing Word Segmentation for Downstream Task (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to optimize tokenizations for downstream tasks are not suitable for traditional NLP. |
| Approach: | They propose a method to explore a tokenization appropriate for a downstream task . they train a model to assign a high probability to such appropriate tokenization based on the downstream task loss . |
| Outcome: | The proposed method improves sentiment analysis and textual entailment tasks . it is also integrated into state-of-the-art contextualized embeddings and reports a positive effect . |
Cross-Topic Rumor Detection using Topic-Mixtures (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on rumor detection models has explored network structures, propagation paths, user credibility and fusion of heterogeneous data. |
| Approach: | They propose a method that adapts a rumor detection model trained on source to target topics to make rumour predictions. |
| Outcome: | The proposed method outperforms baseline debiasing methods in a cross-topic setting. |
Learning to Rank Question-Answer Pairs Using Hierarchical Recurrent Encoder with Latent Topic Clustering (N18-1)
Copied to clipboard
| Challenge: | Existing models for sentence pair ranking are based on hierarchical recurrent neural network and latent topic clustering module. |
| Approach: | They propose a hierarchical recurrent neural network and latent topic clustering module to adapt a recursive hierarchic neural network to rank candidate answers. |
| Outcome: | The proposed model shows small performance degradations in longer text comprehension compared to current models which suffer from it. |
Video Dialog via Progressive Inference and Cross-Transformer (D19-1)
Copied to clipboard
| Challenge: | Existing visual dialog methods use RNN to encode the dialog history as a vector representation . a new method for video dialog is proposed, which progressively updates query information based on dialog history and video content until the agent think the information is sufficient and unambiguous. |
| Approach: | They propose a method which progressively updates query information based on dialog history and video content until the agent thinks it is sufficient and unambiguous. |
| Outcome: | The proposed method can be used to infer video dialog answers on large-scale datasets. |
Hyperbolic Capsule Networks for Multi-Label Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for classification of labels are limited by feature aggregation and encoding. |
| Approach: | They propose to use hyperbolic capsule networks to capture fine-grained label information . they also propose a new routing method to adaptively adjust capsule number during routing . |
| Outcome: | The proposed method significantly improves the performance of multi-label classification on tail labels. |
Debiasing with Sufficient Projection: A General Theoretical Framework for Vector Representations (2024.naacl-long)
Copied to clipboard
| Challenge: | Pre-trained vector representations can inadvertently encode undesirable social biases. |
| Approach: | They propose a framework for reducing bias by transforming vector representations to an unbiased subspace using sufficient projection. |
| Outcome: | The proposed framework mitigates bias across debiasing and fairness tasks and across various vector representation types, including word embeddings and output representations of transformer models. |
Structural Neural Encoders for AMR-to-text Generation (N19-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) graphs are graphs, rather than trees, because they contain reentrant nodes with multiple parents. |
| Approach: | They propose to use sequence-to-sequence models that encode AMR graphs into vector representations to generate sentences from AMRs. |
| Outcome: | The proposed model outperforms tree encoders in the AMR-to-text generation task by 24.40 points. |
Affect-Driven Dialog Generation (N19-1)
Copied to clipboard
| Challenge: | Existing systems for end-to-end dialog generation focus on response quality without explicit control over affective content of the responses. |
| Approach: | They propose an affect-driven dialog system which generates emotional responses using a continuous representation of emotions. |
| Outcome: | The proposed system outperforms existing systems in terms of BLEU score and response diversity, and qualitative measures. |
Exploiting Invertible Decoders for Unsupervised Sentence Representation Learning (P19-1)
Copied to clipboard
| Challenge: | Encoder-decoder models for unsupervised sentence representation learning discard decoder after training . decoded sentences are often used to make better predictions of words in a given sentence . |
| Approach: | They propose two types of decoding functions whose inverse can be easily derived without expensive inverse calculation. |
| Outcome: | The proposed models can learn good representations from encoders and decoders without expensive calculations. |
Multi-View Document Representation Learning for Open-Domain Dense Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval are hard to match with multiple views. |
| Approach: | They propose a multi-view document representation learning framework to generate multiple embeddings through viewers to represent documents and enforce them to align with different queries. |
| Outcome: | The proposed method outperforms recent works and achieves state-of-the-art results. |
Composition, Attention, or Both? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work suggests that language models implicitly learn syntactic structures of natural language, even though they do not receive explicit syntatic supervision. |
| Approach: | They propose a novel architecture that recursively compose subtrees with a composition function and selectively attend to previous structural information with sc-attention mechanisms. |
| Outcome: | The proposed architecture can induce human-like syntactic generalization by recursive composition and selective attention to previous structural information. |
A Black-Box Attack on Code Models via Representation Nearest Neighbor Search (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating adversarial code examples face challenges such as limted availability of substitute variables and the creation of adversarials with noticeable perturbations. |
| Approach: | They propose a search seed based on historical attacks to find adversarial substitutes . they employ a pre-trained variable name encoder to map the search seed to a continuous vector space . |
| Outcome: | The proposed approach outperforms baseline methods in terms of ASR and QT. |
Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that detect semantically shifted words do not account for its evolution through time. |
| Approach: | They propose three variants of sequential models for detecting semantically shifted words . they demonstrate that temporal modelling of word representations yields a clear-cut advantage . |
| Outcome: | The proposed models account for the changes in word representations over time. |
Using Distributional Thesaurus Embedding for Co-hyponymy Detection (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to detect lexical relations among distributionally similar words have been proposed to solve this problem. |
| Approach: | They propose to use distributional semantic models to detect co-hyponymy relations by embedding them into the distributional thesaurus. |
| Outcome: | The proposed model outperforms the state-of-the-art models for binary classification of co-hyponymy vs. hypernymy, as well as co-meronymy by huge margins. |
Open Knowledge Graphs Canonicalization using Variational Autoencoders (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to solve this problem generate embeddings for noun and relation phrases . ambiguous subject-relation-object triples are created by open knowledge graphs . |
| Approach: | They propose a model to learn both embeddings and cluster assignments in an end-to-end approach . they propose CUVA to be able to group noun and relation phrases using embeddable features . |
| Outcome: | The proposed model outperforms state-of-the-art methods over multiple benchmarks. |